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ABSTRACT 

This document is intended to promote understanding of 
performance standcirds in Job Training Partnership Act (JTPA) and 
Employment Service (ES) applications; to promote more appropriate use 
of performance standards in managing and overseeing job training and 
job placement programs at the state and local levels; and to foster 
the integration of the job training and job placement systems through 
performance management. Those purposes are outlined in the first 
section. The second section provides a history of the development of 
performance standards in the JTPA and ES systems and states some 
important principles underlying the approaches developed to support 
performance management in those systems. The third section outlines 
the mechanics of using performance standards. The next section is 
devoted to major criticisms of performance standards, involving both 
conceptual and technical issues. The document's narrative concludes 
by stating some directions for the future and calling for increased 
Coordination between job training and job placement. Appendix A 
presents the key components of the JTPA performance standards system. 
Appendix B shows adjustment model factors for program years 1986 and 
1987. Appendix C is a 12-item annotated bibliography. (CML) 
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Preface 

The National Association of Counties (NACo) is pleased to publish this Issue Paper entitled 
"Performance Standards in Job Training and Job Placement Programs" by Dr. Christopher King. 

Performance standards are among those Job Training Partnership Act program issues which have 
received substantial attention. As this Issue Paper goes to press, the United States Dq)artment of Labor 
has convened a number of task forces and woric groups designed lo review and improve existing 
measures and develq) new ones, if that is necessary. 

We, at the National Association of Counties, believe that this Issue Paper will contribute much to the 
discussion about and proper use of perfomrance standards. The author points out in the first pages of 
this report that it has several purposes, including to promote fuller understanding of, more appropriate 
use of and integration of job training and placement through performance standards. 

This paper was sponsored by N ACo's Training and Employment Programs through a grant from the 
United States Department of Labor Employment and Training Administration. This paper does not 
reflect, necessarily, the views or (pinions of the Department of Labor. 

We would appreciate your comments on this paper. Please address your comments to Neil E. Bomberg, 
Research Associate and Editor, Issue Papers Series, National Association of Counties, 440 First Street, 
N.W., Washington, D.C. 20001. 
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Executive Summary 

The purpose of this report is to promote fuller understanding of performance standards in both Job 
Training Partnership Act (JTPA) and Employment Service (ES) applications, to promote more 
appropriate use of performance standards in overseeing and managing state and local job training and 
job placement j^ograms, and to foster integrated job training and job placement systems through 
performance management 

For more than 20 years, job training and job placement programs have used some type of performance 
management system. Initially, job training programs were funded on the basis of grant proposals and 
past performance records. With the Comprdiensive Employment and Training Act (CETA), funding was 
based on need, rather than performance, though the U.S. Department of Labor (USDOL) did develop a 
technical assistance gwide CTAG) which suggested approaches for ensuring management toward 
performance. In 1978, performance standards were mandated in the CETA Reauthorization Amendments, 
though no rewards or sanctions were included By 1981, USDOL implemented "benchmarks", an 
experimental system of performance standards and related local adjustment approaches, JTPA merged 
funding based on need with funding based on performance. A portion of JTPA funds is set aside and 
awarded based on prior program year performance. Full scale implementation of perfcxmance standards 
in job trailing programs took place in Program Year 1984 and application of sanctions for failure to 
meet or exceed standards for two years occurred for the first time in late 1986. With the completion of 
the first two-year performance management cycle, program accountability via performance standards, 
incentives and sanctions truly had arrived. 

Performance-related responsibilities for the key actors in job placement programs were much more 
loosely defined at all levels. Under Wagner-Peyser, the Secretary of Labor may establish state 
performance standards. To date, the Secretary has established 11 performance measures along with 
corresponding national dq^arture points and a suggested methodology for adjusting four measures. 
Governors may use any or all of the standards and may make appropriate adjustments. Though states use 
some of the standards, none use the adjustment model In addition, some states have distributed 
incentive bonuses; others have not Most importantly, none has used job training and job placement 
performance standards to achieve closer coordination of JTPA programs with the Employment Service. 

Both the job training and job placement pwformance standards systems take into account differences 
among local programs while providng a systematic process for assessing program effectiveness. For 
job training the Secretary is charged with developing and implementing standards. Seven have been 
developed. Their advantage is that they provide a method for determining performance expectations 
based upon past experience , they can be adjusted to account for local conditions, and they offer an 
equitable basis for awarding incentive funds and applying sanctions. 

For job placement programs the underlying rationale for the performance measures is the same as that 
for JTPA. As with JTPA, their advantage is that they provide a method for determining performance 
based on past experience. They can be adjusted to account for local conditions and provide a basis for 
awards or sanctions. 

However, performance standards have been criticized for causing service delivery areas (SDAs) and ES 
offices to become overly concemecd with numbers served, to cream from eligible clients to raise the 
positive termination rates and to move toward quick fix rather than long term solutions. More 
specifically, the adjustment models have been criticized because their explanatory power is relatively 
low; because needed data is limited and difficult to obtain; and because they create performance 
expectations which do not account for chronic over- or under-achievers. 



Efforts to look at post program performance standards are a step to address these issues. Among the 
measures being considered are employment rate 13 weeks after completion of program; average weekly 
earnings; and average number of weeks worked Ultimately, these measures should become the basis for 
integration of JTPA and ES programs to achieve better client services, reduced expenditure of funds in 
these areas, and avoid service duplication. 
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Purpose of the Report 

Since the passage of the Job Training Partnership Act (JTPA, Pi. 97-300) in late 1982, a number of 
technical assistance guides (TAGs) have been developed and distributed to the major actors in the job 
training and job placement systems, many of them dealing with performance standards. With few 
exceptions, these TAGs have been technical in nature, geared toward the staff responsible for the 
day*tOH]ay implementation of the standards provisions and requirements.^ This paper is aimed at an 
audience which includes the staff but also reaches out to encompass a broader groiq), including service 
delivery area (SDA) directors, members and stafTs of the local Private Industry Councils (PICs) and State 
Job Training Coordinating Councils (SJTCCs), and local elected officials. Until the concq)ts and intent 
of performance standards are made understandable to these groups, the true impact of the rew job 
training and job placement directions wiil not become a reality. 

This performance standards paper has several related purposes: 

o To promote fuller understanding of performance standards in both JTPA and 
Employment Service applications. 

o To promote more appropriate use of performance standards in managing and overseeing 
job training and job placement programs at the state and local levels. 

o To foster the integration of the job training and job placement systems through one of 
the more important mechanisms available, performance management 

Although this paper has been designed fcH* use by a very broad audience - including the first-time staffer 
" its primary audience is the SDA or PIC director who, because of necessity, carries a large share of the 
responsibility for program performance within the parameters est blished by performance standards 
requirements. 

Key tqncs covered are as follows: 

o The Context for Performance Standards, including an examination of their evolution and 
important conceptual underpinnings; 

o Perfonpance Standards Mechanics, from the point of view of the informed lay person; 

o Major Problems With and Criticisms Of P^ormance Standards as they have been 
developed and used in recent years; and 

o Future Directions in performance standards and performance management 



Background: The Context for Performance Standards 

This section provides an abbreviated history of the development of performance standards in the two 
systems and elaborates tq)on some of the more important conceptual underpinnings of these provisions. 
With a historical perspective and an understanding of the intent of the provisions, L is paper offers a 
bette-r context for utilizing performance standards. 




A Brief History 

For more than two decades Job training and job placement programs in the United States operated under 
varying arrangements at the federal, state and local levels with only the most cursory attention to 
performance management The following discussion briefly summarizes the evolution of performance 
standards and related provisions in both the job training aiid job placement systems. 

Job Training. Since the emergence of publicly-funded job training programs in the early 1960s, 
there have been three major phases in performance standards implementation. During the first phase 
from 1961 to 1973, programs were inherently performance-based. The federal government provided 
funding for programs at the local level to either governmental bodies or community organizations, 
largely bypassing governors. Funding, however, was typically forthcoming only if the local entity had 
written a quality grant proposal and could demonstrate a record of past performance. 

The passage of the Comprehensive Employment and Training Act (CETA, P.L.93-203) in 1973 marked 
the beginning of the second phase. Here funding followed need rather than performance. From 1973 to 
1983, federal job training funds were allocated based upon measures of relative need, such as shares of 
unemployment or the incidence of the economically disadvantaged population. Shortly after CETA's 
initiation, the U.S> Department of Labor (DOL) issued a series of TAGs designed to help states, cities 
and counties with new program responsibilities associated with implementing a fairly complex set of 
requirements. One of these TAGs was developed around concepts and mechanisms for managing the 
performance of local programs.^ 

There were no particular performance expectations or requirements placed on local efforts beyond 
reporting requirements; the TAG merely suggested appropriate approaches for ensuring management 
towards performance. Regardless of performance, local programs could count on receiving their 
allocations for the year unless th^ had extreme, documented fiscal or legal problems. 

In the mid-1970s there were signs of change. The Department of Labor began to establish goals and 
expectations to varying degrees of specificity, including **performance indicator clusters", in their 
regional offices. Regional DOL rq)resentat!ves assessed the local programs based upon performance 
relative to these indicators. However, there were no positive or negative implications connected to this 
assessment Further, it was left to each individual representative to determine whether and to what 
extent an area's performance indicators needed to be adjusted for differences in local economic 
conditions or participant characteristics through the use of '^interpretive factors". The process was 
highly subjective, in effect making each regional federal representative a walking adjustment model, 
without the quantification or predictability of later £4)proaches. 

Until the 1978 CETA Reauthorization Amendments (Pi. 95-524), performance standards were not 
mentioned in the authorizing legislation for employment and training programs. The 1978 
Amendments introduced performance standards, but were silent on their implications. After a mulci-year 
effort to redesign the nation's management information systems for job training in order to support the 
establishment of standards and subsequent efforts to identify and develop appropriate standards, an 
experimental system of performance standards - known as "benchmarics*" - was initiated in late 1981. 
This system, which was a part of a broader framework for performance management including summer 
youth and post-program standards, was left hanging in the period of uncertainty surrounding the 
phasedown of CETA and the passage of new job training legislation. However, many of the 
components of the current performance standards approach - from the process for developing the 
standards to the local adjustment approach - were laid out in the period immediately preceding the 
passage of JTPA. 

The third and current phase began with the implementation of JTPA. Programs now receive funding 
through a mechanism based upon a combination of need and performance. Most program funding flows 
to local SDAs through a federally-prescribed formula based upon shares of unemployment and 
low-income peculation. A portion of the funds, however, is set-aside and awarded based upon prior-year 
O performance according to standards established by the Secretary of Labor. JTPA, with its accompanying 
IJjC amendments to the Wagner-Peyser Act, which authorizes the fedeial-state labor exchance system, 
initiated the following major changes in the job training^lacement environment: y 



0 Eliminated public sector employment (PSE) from the available tool kit; 

0 Strengthened the role of the governor in planning and managing programs; 

0 Enhanced the control of the private sector in local program planning and operations 
through the private industry councils (PICs); 

0 Mandated closer coordination between the job training and job placement systems 
through joint planning via state job training coordinating councils (SJTCCs) and PICs; 

0 Added a major new funding title, Title in, for aiding woricers displaced due to 
technological changes or laid off due to plant closings or other events; and 

0 Instituted outcome-based performance standards, accompanied by real incentives and 
sanctions, to be established by the Secretary of Labor and implemented by the 
governors. 

Most of these changes in the program environment had considerable precedent The PSE component 
had been zeroed-out of the federal budget in 1980-1981. Governors already had control of the old 
baIance*of-state programs under CETA and varying degrees of influence over their state labor exchange 
efforts. PICs had first been ueated in the 1978 CETA Amendments. Coordination between these two 
programs had been a requirement in some form fcH* a number of years. Programs for displaced workers 
had been operated on a limited, demonstration basis under CETA. The magnified role of performance 
standards and performance management to many observers was the very essence of the newly-constituted 
job training^ob placement system in the JTPA era. 

Full-scale implementation of performance s;andanls took place in Program Year (PY) 1984 (July 1, 
1984 through June 30, 1985). The federal regulations for the 9-month Transition Year 1984 allowed 
governors to reward local SDAs on the basis of performance relative to standards, but precluded them 
from punishing or sanctioning based upon poor performance. Only a few states actually gave out 
incentive awards. With the provision of awards across the country in the fall of 1985 - based on PY 
1984 performance - the reality of these new provisions began to take hold. Application of sanctions 
for failure to meet or exceed standards for two years occurred for the first time in late 1986. With the 
completion of the first two-year perforniance management cycle, program accountability via 
performance standards, incentives and sanctions had truly arrived. 

Job Placement Related developments have been underway in the federal-state job placemeri^ or 
labor exchange system authcmzed throughout the country under the Wagner-Peyser Act of 1933. State 
Employment Service Agencies (SESAs) have been operating under this unique federal-state cooperative 
system for more than 50 years, and, most of this time, they have operated autonomously receiving 
funding directly from the federal government SESAs offer a variety of services - ranging from 
providing labor market information, developing job orders with employers, testing and screening 
potential employees, providing job refeixals, to performing the "woA test" for unemployment insurance 
claimants, wc fare and food stamp recipients and other activities and services - geared to facilitating 
labor exchange between employers and potential employees. One very important distinction which 
should be made between the job training and placement systems is that, while eligibility for job 
training programs is restricted largely to economically disadvantaged persons, local ES offices must 
serve anyone and everyone coming throu^ their doors. Open eligibility is the rule in job placement 
services although some groups such as migrant workers, handicapped persons, and veterans may be 
given special priority for service. 

The performance orientation of the job placement system also has evolved through several distinct 
phases during its lengthy history .3 Focusing upon more recent periods, in line with the discussion on 
job training, there appear to have been more frequent shifts in the performance orientation of these 
efforts, and the phases are more difficult to characterize clearly. In many instances, the orientation of 
the job placement system moved counter to that of the job training system, making coordination 
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something of a problem. During the 1950s and early 1960s, funds for labor exchange were allocated to 
SESAs based upon a combination of factors including measures of staff workload and need, as well as 
performance. This allocation system was dropped in favor of one which distributed funds based upon 
the state's relative share of economically disadvantaged persons in the population, in keeping with the 
redirection of the entire ES system, during the 1960s and early 1970s toward serving the poor. From 
1975 to 1979. funds were allocated to states through a series of formulae based primarily on their 
performance. From FY 1980 to 1982. state funding was primarily tied to its previous year's funding 
level (98 percent). The passage of JTPA. however, brought significant change to the entire structure of 
the labor exchange system, especially performance related aspects. 

Before examining the JTPA Amendments to Wagner-Peyser. it is worth noting thai the only pre-JTPA 
federal provisions related to performance management were those concerning the allocation of funds to 
the states - which only applied to a very small portion of the funds and the general administrative 
requirements that the U.S. Employment Service (USES) prescribe minimum standards of efficiency for 
the system. The USES was also to ensure that relatively uniform management information systems 
were established by the states. Any additional performance management provisions were left to state 
discretion; none were implemented. The entire Wagner-Peyser Act was. and remains, only a few pages 
in length. 

JTPA's passage in late 1982 significanUy altered labor exchange programs. Front-end planning was 
eliminated in favor of the management of performance in relation to outcomes, including 
the relationship to job training efforts. The foltowing changes were particularly relevant to 
performance management: 

o States receive funding based solely upon a needs-based formula which uses relative 
shares of unemployment and the civilian labor force. 

o Funding flows to governors rather than direcUy to SESAs. and state funds are divided 
into two components: 

- Basic Labor Exchange (90 percent) to provide the genera! basic exchange activities 
and services locally through a substatc formula determined by the governor. 

- Govemor's Discretionary Funds (10 percent) for three uses: performance incentives for 
local offices and programs; services for groups with special needs; and exemplary 
models. 

o The Secretary of Labor may establish national performance standards for labor 
exchange, based upon such measures as placements, wages and job retention. 

o Planning for labor exchange must be conducted jointly at the SDA level, with PIC and 
State Job Training Coordinating Council review; thus, coordination between the job 
training and job placement systems is mandated. 

Legislative Provisions and Responsibilities of Key Actors 

The frame woric which has been devised over the last five years for both the job training and job 
placement systems is comprised of most of the same elements and reflects the same basic 
methodological approach. Yet, there are numerous differences in the legislative provisions and 
responsibilities of the key actors in each system. These are now discussed. 

Job Training. The purposes of JTPA are to increase the labor force participation of youth and 
unskilled adults, and to provide job training for economically disadvantaged persons and those facing 
serious barriers to employment [Sec. 2]. The Act views job training as an investment in human capital, 
the return on which for adults is to be measured by: 

9^- o Increased employment and earnings; and j 
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0 Reductions in welfare dependency (Sec. 106 (b)(1)]. 

For youth, the measures are lo include: 

0 Job placement; as well as 

0 Participation in training or apprenticeship programs; 

0 Attainment of employment competencies recognized by the local PICs; and 

0 Completion of elementary, secondary and post-secondary school requirements [Sec. 
106(b)(2)). 

The responsibilities for establishing and implementing these various performance standards provisions 
arc shared by a number of key actors in the job training system. Exhibit 1 briefly describes the 
responsibilities of key actors in both the job training and the job placement systems. The Secretary of 
Labor is responsible for establishing and deflning performance standards for adult, youth and dislocated 
woricer programs. These standards are to be implemented by the governors. To date, DOL has issued 
standards for seven performance measures to be used for evaluating and rewarding performance under 
Title IIA programs. Calculated aimually using prior year JTPA program data, they arc deflned as shown 
in Exhibit 1, on the following page, and Appendices A and B. 

These seven performance measures in place since JlPA*s inception in 1983 and applicable at least 
through PY 1987 (i.e., June 30, 1988) focus exclusively on inunediate program outcomes. However, 
as discussed later, DOL has instituted changes which ultimately will incorporate post-program 
performance measures into the federal performance standards system. Beginning with the PY 1986 
reporting cycle, states are required to collect and repon post-program data on Title HA adult and Title III 
programs. The new rqwrting requirements center around three outcome measures that are used to 
describe the labor maricet experiences of former JTPA program participants over a 13- week period. The 
three measures are candidates for PY 1988 performance standards as potentially better proxies for 
longer-term impacts of program participation. In PY 1988, these proposed measures may be used in 
conjunction with or in place of some or all of the current termination-based measures. 

Six percent of each state*s Title IIA allocation is to be used by the governor to award performance 
through incentive grants to SDAs who exceed their tocdl performance standards and to provide 
generalized technical assistance to all SDAs (Sec. 202 (b)(3)]. Thus, JTPA creates a performance-driven 
system in which SDA-based performance standards are specifled, and performance which meets or 
exceeds these standards is rewarded* In keeping with the decentralized nature of the program, governors 
are ultimately accountable for their JTPA programs and are required to establish policy regarding all 
aspects of the performance standards system - from specifying the methods to be used in calculating 
performance standards for each local SDA to determining the basis upon which six percent funds will be 
awarded for performance against those standards* The relative importance of standards must be 
determined, and the state may consider adding other standards, including those which allow incentives 
for serving participant groups which the state classifles as hard-to-serve. 

In addition, governors are responsible for ensuring that SDAs are utated equitably with regard to 
performance measurement and subsequent determination of eligibility for incentive funds. They must 
initially decide what process(es) will be used for setting the local performance standards, but must 
further clarify and communicate to the SDAs - policy regarding: 
0 Standard setting, adjustment and recalculation procedures; 

0 Incentive grant eligibility and award determination procedures; and 

0 Corrective action or reorganization actions to be applied to SDAs judged as having 
failed to meet their performance standards for two years [Sec. 106 (h)]. 



Exhibit I: Performance Standards RESPONsroiuTiES ofKev^ Actors 


System Actor 


Job Training Title IIA 


Job Placement 


Secretary of Labor 


0 Develops performance standards 
(required). 

0 Establishes parameters within which 
governors may prescribe standards 
variations locally. 

0 Rules on SDA sanction appeals for 
failing to meet standards. 


0 Develops performance standards 
(optional). 

0 Performance standards take into 
account state plan prioriQr differences 
(required). 


Governor 


0 Prescribes variations in performance 
standards for economic, geographic 
and demographic factors, service 
population characteristics and services 
provided. 

0 Establishes and ^plies policies for 

- distribution of SDA incentives 

- provision of technical assistance 

- reorganization/ correction action 
forSDAs. 

0 Establishes additional standards 
(optional). 


0 Adjusts state & substate standards 
for direct or indirect placement 
wages, retention, etc. 

0 Establishes and applies policies fon 

- distribution of incentives to local 
ES offices and programs 

- provision of technical assistance 
(optional) 

- reorganization/ correction action 
(optional). 

0 Review the state ES plan (including 
provisions). 


State Job Training Coordinating Council 
(SJTCC) 


0 Develops the Governor's coordination 
and special services plan. 

0 Recommends to the governor variations 
in performance standards. 

0 Advises on local plan ^proval. 
0 Reviews local perfOTnance. 


0 Advises on local plan approval. 
0 Reviews local performance. 


Local Entities 


0 PICs/SDAs plan and provide for 
services to exceed standards. 

0 PICs develop/recognize youth 
employment competencies. 


0 ES offices plan SDA-level services 
Qointly with PICs and elected officials) 
to exceed standards. 
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Finally, wiih regard to corrective action, states must define what constitutes "failing to meet standards**, 
as well as the sanctions that apply for various types and degrees of failure. Again, the issue of relative 
importance of individual standards should be taken into consideration. Governors* decisions regarding 
SDA sanctions are subject to appeal to the Secretary who is charged with ruling only on whether the 
governor's decision is consistent with the performance standards provisions of the Act. 

Job Placement Performance related responsibilities for the key actors in job placement programs 
are much mc^ loosely defined at all levels. The application of standards occurs in an environment 
which is considerably changed in terms of its planning parameters. With the JTPA amendments, 
planning for job placement has greater local input and tighter coordination with job training, both at 
the local and state level [Sec. 8(b)]. The local FICs, SJTCCs and governors serve major plan review 
functions. 

In sharp contrast to JTPA, under Wagner-Peyser as amended, the Secretary of Labor may establish 
performance standards for states' basic labor exchange programs [Sec. 13(a)]. JTPA requires the 
Secretary to establish standards according to defmite timelines and under a variety of constraints; 
Wagner-Peyser merely authorizes the Secretary to establish them (See Exhibit 1). To date, the 
Secretary has established eleven (11) performance measures, along with corre^Mxiding departure points 
and a suggested methodology fcH* adjusting only four of these measures from the departure points at the 
state and substate (SDA) level. 

The suggested labor exchange performance measures are as follows: 

0 Basic Outcome Measures 

number of individuals placed; 

number of placement transactions; 

number of individuals who obtained employment; and 

number of individuals who secured employment (combination of above two 
measures). 

0 Efficiency Measures 

individuals placed per staff year, 

placement transactions per staff year, 

individuals who obtained employment per staff yean and 

individuals who secured employment per staff year. 
0 Woric Test Measures 

number of Unemployment Insurance (UI) claimants placed; 

• number of UI claimants who obtained employment; and 

number of UI claimants who secured employment 
These measures address different aspects of the labor exchange function. The use of any of them is 
dependent upon the speciflc goals that a state establishes, and the decision to use any of the measures is 
a state prerogative. 

O Governors at the state level are in roughly the same position as the Secretary of Labor. Governors are 
I^C authorized to use any or all of the standards established by the Secretary and to make appropriate 
" adjustments- for state goals, types of programs, local conditions, etc. - at the state and substate level 



tScc. 7(b)(1)]. To date, although several stales have explored the use of the suggested substatc 
adjustment models, none has actually implemented them. Some governors have distributed incentives; 
some have not Very few states have managed to achieve close coordination with job training efforts 
through use of the performance standards process. 

Underlying Principles 

There are a number of important prmcii^es underlying the apjwoaches developed to suppwi performance 
standards and perforaumcc management in the job training and placement systems. These underlying 
princiides include the following: 

o Perfcmnance standards should be used to encourage good performance. Local SDAs 
should not be penalized for trying to do the right thing. Service to gmsps viewed as 
hard-to-serve or most in need should be encouraged rather than restricted. 

o SDAs should be rewarded for good performance, not good planning. 

o SDAs should not be pct talized for operating in a difBcult environment, particularly one 
undergoing major ec gnomic and labor market changes. 

o An SDA should primarily compete against itself, not other SDAs. 

o Performance standards, as calculated through the edjustmc nt nuxlels, should serve as a 
management tool for in program planning, refinement and evaluation. 

J Numerical perfomumce standards should serve as the begiming point (ot state-local 
discussions concerning proper program management, not the ending point 

o There is much more to good program performance than just exceeding a set of 
performance staridards. 

The Mechanics of Performance Standards 
Key Components 

Botfi the job training and job placement perfonnance standards systems take into account differences 
among local programs while providing a systematic process for assessing irogram effectiveness. 
Adjustment methodologies arc available to contrcd for variation among who is served by the program 
and under what kx:al conditions. Further adjustments in JTPA account for the length of participation 
and in ES programs, for resource availability (i.e., staff years paid). In the JTPA system, use of the 
adjustment methodology has received much stronger encouragement from DOL, while in job placement 
programs, the effort has been far more low-key, simply making materials on the niethodology available. 

Job Training* The Secretary of Labor is charged with establishing performance standards and seven 
such standards have been issued for JTPA through PY 1987. The Secretary may only change the 
standards every two years, in keq)ing with the two-year planning cycle. For each of those standards, 
adjustment models are rcestimatcd annually for use at the state and local levels, using actual JTPA data. 
They provide a valid, systematic method for generating performance expectations based upon past 
experience with similar groups of individuals. They offer an equitable basis for awarding incentive funds 
and q>plying sanctions. 

Since the models are developed using prior-year JTPA data and local economic data, they attempt to 
account, or hold harmless, for factors viewed as being beyond the SDA's control, assuming that the 
remainder can be attributed to local managem* it Piiot to PY 1986, the models were specified using 
CETA performance data. When JTPA data (P\ 1984) became available, it was used in the model 
development process. At that time, performance standards were explicidy defined as minimum 



accq>:able levels of performance and were set at levels such thai 75 percent of the SDAs could be 
expected to exceed predicted performance levels for the various outcome measures. 



The performance standards adjustment nKxiels provided by DOL are constructed using a statistical 
technique called multiple regression analysis. This process allows multiple local factors to be 
considered simultaneously and determines the amount of variation in performance - for a given measure 
- that is indq)endently attributable to each local factor contained in the model. (Appendix B presents 
the factors contained in the PY 1986 and 1987 JTPA models.) 

For a local terminee or economic factor to be included in one of the adjustment models, it must: 
o Have a strong relationship to outcomes; 

o Be consistent with DOL policy, (i.c., that SDAs should be held harmless for it); 

o Be objective and easily quantifiable; 

o Have substate data available to suppiKt it; and 

o Not reflect progrsan design and management practices. 

Job Placeme&t The underlying basis for tiie job placement adjustment nKxkls is essentially the 
same as that for the JTPA models, althou^thejc^placementmodelsdonoti^ly toallof the 
eleven performance measures which have been suggested. As with JTPA, statistical techniques have 
been applied to past data on program performance under Wagner-Peyser at the state and substate level 
On eighteen states for which consistent data were available for several consecutive years). The four 
measures for which adjustment models are currently available follows: 

o The number of individuals placed; 

o The number of placement transactions; 

o Individuals placed per staff year; and 

o Placement transactions per s\aSf year. 

Governors opting to use the models for these four measures can be relatively confident that they arc 
employing a fair ^ equitable means of systematically adjusting local performance and, thus, 
^>propriately assessing performance and awarding incentives. The PY 1986-1987 model adjusts the 
departure points for each of these measures for the foUowing factors: 



o 


Unemployment; 


o 


Staff Years Paid; 


o 


Population; 


o 


Population Density; 


o 


UI Claimant Registrants; and 


o 


Youth Registrants. 



It should be noted that "Staff Years Paid** reflects the level of resource availability in the local job 
placement program rather than a management variable, since the substate allocation of staff is 
O primarily a state-level decision variable, in sharp contrast to the JTPA system. Like JTPA, the job 

J^C placement adjustment models* ftmciion is to adjust for factors beyond the control of local management 



PY 19i FTP A Performance Standards Calculations 

Governors arc required to set local SDA standards, using established procedures, for each of the 
federally-mandated pcrfonnance measures. For PY 1987. the states have several options regarding 
performance standard calculation. They may opt to: 

o Use the Department's adjustment models developed for PY 1987 ^plication; 
o Use the PY 1986 adjustment models; 

o Adopt and apply the national standards to all SDAs in the state; or 

o Develop another approach for standard selling and measurement, consistent with criteria 
established by DOL 

DOL policy allows states to use either the PY 1986 or PY 1987 models for PY 1987 largely to allow for 
consistency over the two-year (PY 1986-1987) plaining cycle and to fc-lp minimize disnipUons 
associated with implementation of the new federal post-piogiam reporting requirements. In addition, it 
IS hc^ed that this choice - accompanied by the governor's option to make additional adjustments to 
local performance standards -will encourage additional services to client groups viewed as at-risk or 
hard-to-serve, who typk:ally require longer-term, higher-cost services. 

Slates have the option to make further adjustments to local standards, provided such adjustments meet 
ihe parameters set forth by DOL (Sec Appendix A). ItshouMbenoiedthatuseofeitherof iheDOL 
models minimizes the states* burden for development and adjustment, since each of the factors included 
m the models meets the Secretary's parameters. Thus, only additional adjustments would necessitate 
further documentation, Inaddition.incascof SDA appeals, the Secretary is predisposed to ruling in 
favor of states using the Dqwutment's adjustment methodotogy. 

It is essential to remember that the predicted pt^fonnance standards arc basically a moving target 
They are estimated during die planning process ai Uie beginning of die program year using planned 
temiinee characteristics. However, the predicted level changes daily as participants move through the 
delivery system and terminate from Uie local employment and training programs. An SDA's fmal 
performance standard is Uie one calculated at the end of the year, using actual terminee and program 
information, along with updated local economic data. The recalculated number, as estimated via the 
model or further adjusted by the state, serves as the benchmark for detemiining whether <x not the 
SDA has exceeded its standards. 

PY 1986-1987 Wagner-Peyser Standards Calculations 

The process of calculating local pcrfomiance standards under Wagner-Peyser programs is parallel to tiiat 
under JTPA, at least for those measures for which the adjustment meUiodoIogy is available. Planned data 
for die upcoming program year is fed into the relevant model to give planned performance standards. 
These standards account for most important local variations not subject to management decision, 
including population and economic conditions, resource availability and the types of individuals likely 
to be served. They do not adjust for such factors as q)ecial k)cal policy emphasis or odier exceptional 
circumstances not provided for in the model. Examples of special policy emphasis included local 
projects to increase the level or type of service to ex-offenders, those with language barriers, or other 
hard-to-serve groups not already factored into the models. Exceptional circumstances - which might 
have either positive or negative effects on expected performance --might include major business 
opening/closing, labor disputes, the development of exclusive hiring agreements with major 
employers, and natural disasters, among others. 

At the end of the program year fmal performance standards can be calculated based upon actual events 
taking place during the year, producing a "moving target" phenomenon similar to that in JTPA. The 
^ end-of-year calculations may account for at least some of die above mentioned, exceptional 
ERiC ^^"'"stances. Governors may also want to ^ly odicr types of adjustments to the calculated 
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performance standards, including adding productivity factors, specific past performance adjustments 
(to adjust for local programs which are persistent loworhighperfOTners relative to predictions 
based on the models). 

Aside ftom purely technical issues, the processes for adjusting local perfomumce under both the job 
training and job placement systems are remarkably similar. The primary differences lie in the fact 
that models are not available for all of the job placement measures and that the specific factors in 
the models differ. The underlying principles, most of the responsibilities at each level, and the 
calculation steps and their sequence arc the same. Adoption of the DOL-rcconunended performance 
standards adjustment methodologies for both JTPA and Wagner-Peyser would go a long way 
toward facilitating flic coordination and integration of service delivery between two offlie major 
human resource development systems in a state. 

Major Criticisms of Peformance Standards 

When considering the major problems and criticisms associated wifliUieperfonnance standards 
system, it is essential to differentiate between general conceptual problems affecting flie 
system and specific technical shortcomings or limitadons related to flic adjustment models provided by 
DDL for slate and local use. The discussion which follows is focussed upon flic JTPA system. There 
has been little critical discussion of flie Wagner-Peyser system for performance standards and its 
accompanying adjusunent meUwdology. largely because it has not been put to flic test of widespread 
usage. 

Conceptual Issues 



During flie development and early implementation jAases of performance standards, several 
criticisms have been levied against various system attributes, including: 

o Too Numbers Focused. The aim of instituting performance standards is to create a 
performance-driven system which bofli encourages and rewards good performance. 
i.e.. appropriate service to eligible participants. However, it can be argued that such a 
system causes too much attention to be focused on simply hitting flic numbers - at 
flie possible expense of program quality. 

0 Creaming from tfie Eligible Population. Section 141 of flie Act established a 

dichotomy r^arding flie targeting of program services by calling (or flie provision 
of "...employment and training opportunities to fliose who can benefit from, and 
who are most in need of. such opportunities,.,.** By focusing service delivery 
largely on fliose individuals who are more job ready, program operatcns can 
substantially improve flieir (gross) program outcomes. Such individuals, considered the 
''cream" of flie eligible population, usually require less intensive, shorter-term 
program treatments, while offering greater chances for immediate job placement or 
oflier positive outcomes. Whether achieved by outreach, recruitment, screening, 
assessment activity assignment procedures, creaming of flie eligible population 
offers a fast return on investment, translating to quick program successes in tenns of 
increased placements at lower costs, 

o Program Design Options, The decentralized nature of JTPA provides SDAswifli a great 
deal of flexibility to plan and operate local job training programs. As long as flie 
performance standards are geared toward termination-based, rather than post-program, 
measures, local program operators can continue to achieve higher performance 
numbers. Such choices may or may not meet flie immediate employment-oriented 
goals of most participants, yet may sacrifice flie chances for longer-term job and 
^ employment retention, wage gains and promotional opportunities, 

ERIC 

Quick-fix options may be utilized in order to play flie numbers game, and may not meet flie needs 
of many participants. Examples of typical program design trade offs include: 



- Increased use of short-term job search assistance activities rather than 
longer-term skills training which resulUng in more placements at lower cost 

- Decreased emphasis on GED and English as a Second Language (ESL) programs for 
adults, which may be essential to ensure retention, and possibly attainment of quality 
jobs due to cost and because if no placement occurs. compleUon of such worthwhile 
and necessary acUviUes counts as a non-posiUve outcome under the current Tide IIA 
standards framew(Hk. 

Program choices of the type and length of training to be offered influence the selecUon of target groups 
to be seived and vice versa. Tliepolicy dlrecUon ofthc PIC. along with the program design 
choices of SDA management staff, deteimine the direcUon a program wiU take. Choices maybe 
limited by many factors, including availabiUiy of funds and other resources, lack of qualified service 
providers, need of the eligible popuIaUon as weU as needs of the employer community Standards 
areprcscnptivc by nature, however, they should be viewed as benchmarks which play a role in 
program design, not as the controlling factor. 

Ulumatcly. local program operators arc responsible for making management decisions. i.e.. for 
making ^jjwpriate program choices designed to meet the job and skill demands of employers in 
their local communiUes. Such decisions are made more challenging by the effects of changing 
economic and labor market ccmditions. 

Technical Issues 

Criticisms associated with the performance standard adjustment models include: 

0 Explanatory Power. Even considering that the models do not attempt to control for all 
factors related to performance predictions, the explanatory power of some of the 
models is relatively low. It may be that the models which have been developed 
attribute too much to local management and do not control sufficiently for 
non-management factors. 

o Data Limitations: Economic. Up-to-date local economic data for use in the 

standards adjustment process is difficult to obtain. This is particularly important in 
those SDAsand areas in which large plant shutdowns or other similar events have 
occurred. (Note tiiat the governor may make allowances for just such situations at 
year's end. if an adjustment is requested.) In addition, differences often exist between 
the geographic boundaries of an SDA and the applicable labor maricet areas. 

o Data Limitations: Participant Several problems exist regarding the participant 
(terminee) data consid^ed by the models: 

Terminee data cannot account for definitional differences including youth 
employment competency definitions - prevalent in local and state reporting; 

Data considered for inclusion is limited to data available for all SDAs. i.e.. 
basic demographic infamation; and 

It is difficult to assess the degree of importance of hard-to-serve or other groups 
for which factors are not explicitly included in the model. 

o Performance Expectations. By defining minimum acceptable levels ofperfomance. 
the model does not account for chronic over- or under-achievers. 
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Directions for the Future 



Post-Program Standards 

The Department's move to implement post-program performance standards represents an effort to 
fulfill the original intent of the Act. As previously mentioned. Section 106 refers to JTPA-fmanced 
job training efforts as an investment in human capital. The development of post-program 
measures - working alone or in conjunction with the current termination-based measures - 
will provide a set of criteria for measuring the return on such investments. 

The Act further states that the return on investment is to be measured by increased employment and 
earnings of participants and by reductions in welfare dependency. DOL has identified three candidate 
measures for which post-program data collection began in PY1986. Implementation of any such 
standards is targeted for PY 1988. It should be noted that DOL does not forsee simply adding 
post-program standards to the existing seven standards. It is much more likely that one or more of 
these candidate post-program standards will be substituted for existing ones. Further, it should also be 
noted that the law does not require that exactly seven standards be established. DOL could just as 
easily choose to implement fewer than seven at some point in the future. 

The candidate measures, which relate to terminee experiences during the 13-week period following 
receipt of program services, are defmed as follows: 

0 Employment Rate (at follow-up): The percentage of total respondents (those with 
completed interviews) who were employed during the 13th fuU postptognun week; 

0 Average Weekly Earnings of Employed Respondents at follow-up: The sum total of 
gross (before tax, health and other deductions) weekly earnings of employed 
respondents during the 13th postprogram week, divided by the total number of 
respondents employed during the 13th week; and 

o Average Number of Weeks Worked (in the follow-up period): The sum of the number 
of weeks worked during the full 13-week postprogram period by all respondents, 
divided by the total number of respondents. 

Youth Program Goals and Objectives, 

Although youth standards are mandated in JTPA's Title II A adult and youth programs, no explicit 
references to performance measurement in the Title IIB summer youth programs were made until passage 
oftheJTPA amendments of 1986 (P.L. 99-496). The amendments further clarified the purposes of 
the summer program as being to enhance basic educational skills, encourage school completion or 
enrollment in alternative school programs, and provide exposure to the world of woric [Sec. 251]. 
For the first time, local service providers are required to assess participants' reading and math 
skills and to establish written program goals and objectives for use in evaluating program effectiveness 
[Sec.253]. 

To date, no federal regulations or further guidance has been issued by the Department as states and 
SDAsmoveto meet the new requirements. The amendments do specify four'areas - tied directly to the 
newly-stated program purpose - which may be covered by the goals and objectives [Sec.2SS]: 

o School retention and completion; 



o Academic performance, including reading and math comprehension; 



o Employability skills; and 



Title III Performance Standards 



Section 106(g) of JTPA requires the Secretary to prescribe Title III performance standards based on 
placement and retenUon in unsubsidized employment DOL has yet to establish Tide III 
standards or adjustment models similar to those provided for TiUeHA progi-ms- however 
governors are required to set entered employment rate standards and are encouraged tose't cost per 
entered employment goals for all Tide m formula-funded pn)^^^ but not for those financed wiS, 
Ae Secretary s discrcuonary TiUe III funds. In addiUon. a naUonal entet^ employment goal of 
60 percent IS specified. & 

The oveniding problem hindering the development and implementaUonofTiUe III standards 
lies ui the Federal reporting requirements. Unlike the TiUe IIA requirements, which call for data 
reporting at the SDA-Ievel. states must only provide TiUe III informaUon at the state, rather than 
local project, level. This results in the compilation of a data base which is inadequate for use in 
developing adjustment models like those used by TiUe DA programs. Although post-program 
data is now bcuig collected for former TiUe III program parUcipants. Uie problem conUnues since 
the new requu^ments also apply only to statewide reporting. 

In Conclu.sion! Tnrm^gpH Tnh Tn^"^" g/Job Placement Coorditifltmn 

In recent years, there has been considerable debate about Uie fumre shape and direction of Uie 
nauon's labor exchange system. Important issues have been raised conceming Uie appropriate roles 
and functions for Uie federal and state governments, as well as flieir financing. In Uiis era of tight 
budgets and reduced resource availability at all levels of govemment. it is apparent Uiat 
increased coordination between JTPA. ES and other human resource development programs will 
Sto^'Tc ^ ^ ^"'P'^'^ed. WheUier Uiis attention to coordination will include closer linkages in 
JTPA/ES planning and performance management, in order to achieve more cost-effective program 
interventions, remains to be seen. 



Performance Standards in 
Job Training and Job Placement Programs 

Footnotes 

1 For example, sec: US Department of Ubor. Employment and Training Administra^ 
Setting JTPATitUIIA Performance StandardsforPY 87 (Washington, DC: March, 1987); 
Measuring'Up: Planning and Managing with Performance Standards - PY 86 (Washington, DC: 
National Alliance of Business. 1986) and the companion piece. Planning and Managing Mdth 
Performance Standards: A Guide for PIC Members (Washington, DC: National Alliance of Business, 
1986); and Beyond the Model: An Approach to Negotiating JTPA Performance Standards, Technical 
Assistance Guide (Washington, DC: National Association of Counties, February, 1986). 

2 US Department of Labor, Manpower Administration, Comprehensive Employment and Training Act of 
1973: Program Assessment Guide CWashxngtDV.DC: US Government Printing Office, 1975). 

3 This discussion relies upon two reports prepared by Abt Associates, Inc. Abt Associates, Inc. has 
performed much of the developmental work on performance standards for Wagner-Peyser programs over 
the last 10 years. These include: Charles K..FdiTCh\ld, A Perforrnance and Needs Based Methodology for 
Allocating Employment Ser}dce Grants: Final Report (Cambridge, MA: Abt Associates, Inc., April 30, 
1980) and Charles K. Fairchild, Development of Employment Service Performance Standards for 
Substate Areas: Final Report (Washington, DC: Abt Associates, Inc., September 30, 1985). 
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Appendices 
Appendix A 

Key Components of the JTPA Performance Standards System 

Factor Weights. For each local factcx^ included in a given performance standard 
adjustment modeI» the multiple regression analysis computes a weight which defines the direction 
and extent of its impact on the standard The weights are used to calculate the incremental change- 
positive or negative- in the performance standard that is caused by a change in the value of the 
local factor. 



Local Factors. The local factors (i.e.» terminee characteristics, average weeks of participation, 
local economic factors) are calculated for the given SDA for use in adjusting the national 
departure point or benchmark. The adjusunent varies by the weighted amount of the difference 
between the local and national average values. The local factors vary for each of the seven models, 
and are included only if they are statistically significant 

National Averages. Calculated for each local factor included in the performance standard models, 
the national averages represent the average value - across allSDAs- for the most recent year for 
which complete data are available. 

National Departure Points. Hiis value serves as the benchmark from which local SDA 
performance standards are derived. It is adjusted upward or downward by the amount of variance of 
SDA local factor values from the national averages. The departure points used in the PY 1986/87 
models aie listed below. With the exception of the Youth Positive Termination Rate, which now 
accounts for the inclusion of youth employment competencies, all are set at the 2Sth percentile. 

National Standards. For the seven performance measures, the numerical national standards 
serve as performance goals for the national JTPA system and may serve as local standards in 
states opting not to use the adjusunent methodology. The PY 1986/87 national standards are 
equivalent to the rounded departure points used in the adjustment models. 

Performance Meastire Point Standard: Departure National 



Adult Entered Employment Rate (AEER) 
Adult Cost per Entered Employment (ACEE) 
Adult Average Wage at Placement (AAWP) 
Adult Wdfare Entered Employment Rate (AWEER) 
Youth Entered Employment Rate (YEER) 
Youth Positive Termination Rate (YPTR) 
Youth Cost per Positive Termination (YCPT) 



62.4 % 
4374.00 
4.64 
51.3 % 
43.3 % 
74.8 % 
3711.00 



62.0 %^ 
4374.00^ 
4.91^ 
51.0 %^ 
43.0 %^ 
75.0 %c 
4900.00^ 



^Set at the lowest quardle, with 75% of the SDAs expected to meet or exceed this level of 
perfonnance. 

^Equivalent to thePY 1984/85 sumdardto encourage quality placements. 

Policy decision to set at this level - CETA numbers high tnd PY 1984 JTPA low - due to the 

inclusion of youth employment competencies. 
^Policy decision to set over PY 1984 actual in order to allow flexibility in program service 

provision. 

Perfonnance Standard. The minimum acceptable level of performance for a speciflc measure of 
program outcomes. 
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Secretary's Parameters for Standard AdiQs»m..nf r„ 



0 Procedures must be: 



responsive to the intent of the Act; 

consisienUy applied among SDAs' 

objective and equitable throughout the State; and 

in confonnance with widely accepted statistical criteria. 



0 Source data must be: 



of public use quality; and 
available upon request 

0 Results must be: 

clearly documented; and 
reproducible. 

0 Adjustmem factors must be limited to: 

economic factors; 
labor market conditions; 

- characteristics of the population to be served; 
geographic factors; and 

- types of services to be provided. 
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A ppendix B 

Program Year 1986 and 1987 
Adjustment Model Factors 



Adult Models • 


AEER» 


ACEE 


X 


AAWP 


AWEER^ 


Program Year: 






1986 


1987 




1QR7 


1986 1987 


Tcnnince 
















CharicUristics: 
















% Female 


X 


X 


X 


X 


X 


X 




% 55+ years 


X 




X 




X 




X 


% Black 


X 


X 


X 


X 


X 


X 


X X 


% Hispanic 


X 


X 


X 


X 






X 


a Native American 


X 












X 


% Asian 




X 


X 


X 


X 




X 


% Dropout 


X 


X 


X 






X 


X X 


% Handicapped 


X 


X 






X 




X X 


% UI Claimant 


X 


X 


X 


X 


X 


X 




% Welfare Recipient 




X 


X 


X 


X 


X 




Program 
















Characteristics: 
















Average Weeks 
















Enrolled 








X' 


X 






Local Economic 
















Characteristics: 
















Average Area Wage 


X 




X 




X 






UI Rate 


X 


X 


X 


X 




X 


X 


% Families Below 
















Poverty 


X 






X 


X 






Population Density 


X 


X 




X 


X 


X 


X 


Employee/Resident 
















Worker Ratio 


X 








X 




X 


Average Annual 
















Earnings 
















Wholesale/Retail 
















Trade 


X 




X 




X 




X 



*AEER: Adult entered employment rate 
^ACEE: Adult cost per entered employment 
^A AWP: Adult average wage at placement 
*AWEER: Adult welfare entered employment rate 
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y oimi Models 

Program Year: 



Tcnnincc Characteristics: 

% Female 

% 14 - 15 years 

% 18 - 21 ycare 

% Black 

% Hispanic 

% Native Americans 

% Asian 

% Dropout 

% Student 

^ High school graduate 
% Post High School 
Attendee 
^ Handicapped 
% Offender 
% Welfare Recipients 



Program Design: 
Average Weeks Enrolled 



Local Economic 
Characteristics: 



Average Area Wage 
Unemployment Rate 
Population Density 
Average Annual Earnings 
Wholesale/Retail Trade 



YEER' 



1986 



X 
X 
X 
X 
X 
X 

X 
X 



X 
X 
X 
X 



1987 



X 
X 
X 
X 
X 
X 



1986 



1987 



X 

X 

X 
X 
X 

X 
X 



X 
X 
X 
X 



X 

X 
X 



X 
X 



X 
X 



YCPT^ 



1986 



1987 



X 




X 


X 


X 


X 


X 


X 



X 
X 
X 
X 

X 
X 
X 
X 



X 
X 
X 



X 
X 



i°"^^«^^<^d employment rate 
3YrS' 3°"?P^5^^^^terminaUonraie 
VCPT, Youth cost per positive temiination 
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An Annotated List of Selected 
Performance Standards References 

JTPA 

Guide for Setting JTPA Title IIA Performance Standards for PY 87 y 

Washington, D.C.: Department of Labor, Employment and Training Administration, 
Office of Strategic Planning and Policy Development, March 1987. 264 pages, including 
121 pages of appendices. 

This eight-part report is the official Department of Labor guide to JTPA Title HA 
performance standards for PY 1987. It is oriented to state-level staff but useful to local 
PIC/SDA (and service provider) staff as well. The guide presents standards in the context 
of performance management systems and details the current national standards, contrasting 
them with the departure points used in Departments optional performance adjustment 
model.. Use of this optional model by governors in making SDA-level adjustments to the 
seven standards is strongly encouraged in the guide, with sections on the development of the 
model, the adjustment calculation steps, and s^pendices containing the woricsheets and 
local economic data. New sections include options for governors to make 
beyond-ihe-modcl adjusttnents and means for setting goals in performance-based contracts. 
Note: The comparable Title IIA Guide for PY 1986 is also available and still appropriate, 
since the Department of Labor supports the use of either the PY 1987 or PY 1986 models 
during PY 1987. 

For information contact: Karen Greene, U.S. Department of Labor, Employment and 
Training Administration, Room N-53 10, 200 Constitution Avenue, N.W., Washington, 
D.C. 20210(202/535-0687). 

Development of Adjustment Models for PY 86 JTPA Pefformance 

Standards: Final Report, MenloPark California: SRI International, June 1986. 

Prepared for the U.S. Department of Labor by Richard W. West and Katherine P. Dickinson. 

145 pages. 

This technical report details the development of thePY 1986 performance standards 
adjustment models, including secticHis on the choice of the underlying da models which 
were examined^ and options considered for model departure points. The report also offers a 
section on possible beyond-the-model adjustments and a brief discussion of Title III 
standards and modeling issues. Note: The technical report for the PY 1987 model 
develq)ment process will be available from the Department of Labor later this year. 

For information contact: Richard WestorKathy Dickinson, SRI International, 333 
Ravenswood Avenue, Menlo Park, California 94025(415/325-6200). Copies are also 
available through the Department of Labor; contact Karen Green at the Department of 
Labor (see above). 

Beyond the Model: An Approach to Negotiating JTPA Performance 

Standards. Washington, D.C: National Association of Counties, National 
Governors* Association and National Association of Private Industry Councils, February 
1986, 77 pages. 

Prepared as a joint effort by three major public interest groups. Beyond the Model is 
aimed at state and local program, administrators working in the JTPA performance standards 
system. It presents an approach to negotiating local performance standards that will 
be reflective of local needs and conditicms, and that can be used as a program management 
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performance reports which PICs may wish to use in conjunction with orientation to 
performance standards and performance management particularly for new PIC members. This 
guide is a companion to the more detailed guide, Measuring Up, described below. 

For information contact: NAB Qearinghouse. 1015 15th Street, N.W., Washington. D.C. 
20005 (202/289-2910) or any of the seven NAB Regional Service Offices. 

Measuring Up: Pkmmng and Managing with Performance Standards^ PY 86, 

Washington, D.C: National Alliance cf Business, 1986. Prepared by Steven Golightly and 
Terence Singer, based upon an eariier version developed by the Employment and Training 
Instiuite, Inc. of New Jersey. 45 pages. 

This is the more technical companion piece to the guide discussed above. Following a brief 
overview of the evolution of and rationale supporting the PY 1 986 JTPA standards, this PIC 
oriented guide presents a systematic sqjpcoach to cdlecting and analyzing performance data 
on activities aiui contractors locally, complete with a series of work^ieets. It also provides 
woricsheets to enable comparison with other SDAs. The guide offers two possible 
approaches for PICs to take in re^xmse to identified perfonnance problems: onetoassist 
with existing systems and contractors, the Oversi^ Reqx>nse; and the other to help with 
new approaches, the Planning Re^KMise. An IBM compatible software package, with annual 
updates, is available to accompany the guide ($349 for the software, $50 per update). 



For infwmation contact: NAB Qearinghouse (see above). 

FoUoW'Up Technical Assistance Guide for PosiProgram Data Collection Under 

the Job Training Partnership Act, Washington, D.C: U.S. Department of Labor, 
Employment and Training Administration, Version 3.0, June 1986. Plrepared by Christopher 
T. King, Steve Marcus, and Laura L. Pittman. 137 pages. 
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This guide is the official Dq>artment of Labor guide to the new data collection and reporting 
requirenients for JTPA Title IL\ and Title III which went into effect in October 1986. The 
guide offers suggestions on organizing and operating a postprogram data coUection effort, 
provides detailed procedures to follow in contacting and interviewing former participants, 
and includes sections on adjusting for re^nse rate bias and validating interview data. The 
appendices include sample forms used in several states, references on survey research, etc. 

For inforoaiion contact: Karen Greene, U.S. Dqjaitment of Labor (see above). 

Job Training Partnership Act Pefformance Standards for Reductions in Welfare 
Dependency: Technical Assistance Guide, V/dshinffon, D.C: Technical 
Assistance Guide, U.S. Department of Labor, Employment and Training Administration, 
OfficeofStraiegicPIanningandPoIicy Development, September 30, 1985. Prepared by 
Deborah Kogan, Nancy Dickinson and David Drury of Berkeley Planning Associates. 75 
pages. 

This guide is directed to both states and local PICs/SDAs interested in pursuing either the 
establishment of optional welfare reductions standards beyond those required by the 
Department of Labor or estimating the impact of JTPA on welfare case closures or grants. It 
describes three distina alternative approaches which can be used: direct measurement of case 
closures and grant reduction outcomes using client-level welfare agency records; development 
and use of a statistical (regression) model to estimate closures and grant reductions based 
upon past relationships; and application of case closure and grant reduction formulas (to 
JTPA outcomes data) derived from welfare agency adminisiraiive rules. Because of the wide 
variation in welfare programs between states and localities, the guide necessarily constitutes 
a useful beginning point for developing such approaches. It also provides a brief discussion 
of the steps necessary to estimate net, rather than gross, welfare savings. 
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